Convergence of stochastic approximation via martingale and converse Lyapunov methods

In this paper, we study the almost sure boundedness and convergence of stochastic approximation (SA) algorithm. At present, most available proofs are based on ODE method, iterations is an assumption not a conclusion. Borkar Meyn (SIAM J Control Optim 38:447–469, 2000), it shown that if has only one globally attractive equilibrium, then under additional assumptions, bounded surely, SA algorithm converges to desired solution. Our objective in present paper provide alternate proof above, martingale methods, which simpler less technical than those method. As prelude, prove new sufficient condition for global asymptotic stability ODE. Next “converse” Lyapunov theorem existence suitable function with Hessian, exponentially stable system. Both theorems independent interest researchers theory. Then, using these results, conditions We show through examples our theory covers some situations covered by currently known specifically (2000).